Burst-level congestion control using hindsight optimization

We consider the burst-level congestion control problem in a communication network with multiple traffic sources, modeled as infinite banks of fluid traffic. The controlled traffic shares a common bottleneck node with highpriority cross traffic modeled as Markov-modulated fluid. We introduce a simulation-based congestion control scheme capable of performing effectively under rapidly-varying service rate by making use of the stochastic model of the cross traffic. In the scheme, the control problem is posed as a finite-horizon Markov decision process and is solved heuristically using a technique called Hindsight Optimization. The goal is to maximize throughput minus scaled delay at the bottleneck node. Our empirical study shows that the control scheme performs significantly better than conventional congestion control methods. Keywords—communication networks, congestion control, traffic model, Markov-modulated fluid, Markov decision processes, simulation.